This position paper discusses the problem of multilingual evaluation. Using simple statistics, such as average language performance, might inject linguistic biases in favor of dominant language families into evaluation methodology. We argue that a qualitative analysis informed by comparative linguistics is needed for multilingual results to detect this kind of bias. We show in our case study that results in published works can indeed be linguistically biased and we demonstrate that visualization based on URIEL typological database can detect it.

In manufacturing, the production is often done on out-of-the-shelf manufacturing lines, whose underlying scheduling heuristics are not known due to the intellectual property. We consider such a setting with a black-box job-shop system and an unknown scheduling heuristic that, for a given permutation of jobs, schedules the jobs for the black-box job-shop with the goal of minimizing the makespan. Here, the jobs need to enter the job-shop in the given order of the permutation, but may take different paths within the job shop, which depends on the black-box heuristic. The performance of the black-box heuristic depends on the order of the jobs, and the natural problem for the manufacturer is to find an optimum ordering of the jobs. Facing a real-world scenario as described above, we engineer the Monte-Carlo tree-search for finding a close-to-optimum ordering of jobs. To cope with a large solutions-space in planning scenarios, a hierarchical Monte-Carlo tree search (H-MCTS) is proposed based on abstraction of jobs. On synthetic and real-life problems, H-MCTS with integrated abstraction significantly outperforms pure heuristic-based techniques as well as other Monte-Carlo search variants. We furthermore show that, by modifying the evaluation metric in H-MCTS, it is possible to achieve other optimization objectives than what the scheduling heuristics are designed for -- e.g., minimizing the total completion time instead of the makespan. Our experimental observations have been also validated in real-life cases, and our H-MCTS approach has been implemented in a production plant's controller.

临床编码是将患者健康记录中的医疗信息转换为结构化代码的任务，以便它们可用于统计分析。这是一项认知且耗时的任务，遵循标准过程，以达到高水平的一致性。自动化系统可以支持临床编码，以提高该过程的效率和准确性。我们介绍了自动临床编码的想法，并从人工智能（AI）和自然语言处理（NLP）（NLP）的角度总结了挑战，该文献是根据文献，我们在过去两年半（2019年末 - 2022年初）的项目经验），以及与苏格兰和英国的临床编码专家的讨论。我们的研究揭示了应用于临床编码的当前基于深度学习的方法与现实世界实践中的解释性和一致性之间的差距。基于知识的方法代表和推理了标准，可以解释的任务过程，可能需要将其纳入基于深度学习的临床编码方法中。尽管面临技术和组织的挑战，但自动化的临床编码是AI的一项有前途的任务。编码人员需要参与开发过程。在未来五年及以后，开发和部署基于AI的自动化系统需要实现很多目标。